Skip to content

fix: avoid stale process slots after compaction - #242

Open
AnatoliyKizyulya wants to merge 2 commits into
Project-HAMi:mainfrom
AnatoliyKizyulya:fix/process-slot-compaction
Open

fix: avoid stale process slots after compaction#242
AnatoliyKizyulya wants to merge 2 commits into
Project-HAMi:mainfrom
AnatoliyKizyulya:fix/process-slot-compaction

Conversation

@AnatoliyKizyulya

@AnatoliyKizyulya AnatoliyKizyulya commented Jul 30, 2026

Copy link
Copy Markdown

Fixes #241

Summary

Fix stale cached process slots after shared-region compaction and avoid
unnecessary status waiting when SM limiting is disabled.

Changes

  • Clear the source slot after moving the last live slot during compaction.
  • Validate cached slots against the active process-slot range.
  • Fall back to an active-slot lookup when the cache is stale.
  • Skip status waiting when no SM limit is active.

Validation

  • CUDA hook consistency check passed: 211 entries and 217 wrappers.
  • Build succeeded in an isolated CUDA 13.3 environment.
  • Validated on NVIDIA RTX PRO 6000 Blackwell Server Edition (SM 12.0),
    driver 580.159.03.
  • vLLM 0.25.1, TP=2, Qwen3.6-27B-NVFP4 remained stable under load with
    CUDA_DEVICE_SM_LIMIT=0 and CUDA_DEVICE_SM_LIMIT=50.

AI assistance was used for drafting; the change and validation results were reviewed by the author.

Summary by CodeRabbit

  • Bug Fixes
    • Improved process-slot validation to prevent stale information from affecting GPU status updates and memory accounting.
    • Refined cleanup of inactive process slots to improve the accuracy and reliability of resource tracking.
    • Updated synchronization behavior so waits and retries occur only when GPU core limits are enabled, reducing unnecessary blocking when limits are not in use.
    • Improved handling of per-device GPU limits for more consistent resource management.

@hami-robot

hami-robot Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: AnatoliyKizyulya
Once this PR has been reviewed and has the lgtm label, please assign archlitchi for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hami-robot
hami-robot Bot requested review from archlitchi and chaunceyjiang July 30, 2026 14:32
@hami-robot

hami-robot Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Welcome @AnatoliyKizyulya! It looks like this is your first PR to Project-HAMi/HAMi-core 🎉

@hami-robot hami-robot Bot added the size/L label Jul 30, 2026
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The change validates cached process slots after shared-region compaction, updates memory and status paths to use valid slots, clears moved slots atomically, and gates readiness waiting on enabled per-device SM limits.

Changes

Process Slot Safety and Waiting

Layer / File(s) Summary
Atomic process-slot compaction
src/multiprocess/multiprocess_memory_limit.c
Slot removal copies the last active slot, atomically clears the source slot, and updates region_info.my_slot when required.
Validated current-slot access
src/multiprocess/multiprocess_memory_limit.c
Status updates, status reads, and GPU memory accounting validate the cached slot by address, active range, alignment, and PID.
GPU core-limit wait gating
src/multiprocess/multiprocess_memory_limit.c, src/multiprocess/multiprocess_memory_limit.h, src/include/memory_limit.h
Adds is_gpu_core_limit_enabled() and makes ENSURE_RUNNING() wait only when a partial SM limit is enabled.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Suggested reviewers: archlitchi, chaunceyjiang

Poem

A rabbit checks each shared slot,
Clears moved entries on the spot.
Valid status paths now align,
And waits obey the SM limit line.
Hop, safe memory, hop!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the primary fix for stale process slots after compaction.
Linked Issues check ✅ Passed The changes address all coding objectives in issue #241, including slot clearing, range validation, stale-slot lookup, and conditional status waiting.
Out of Scope Changes check ✅ Passed The changes remain within issue #241 and directly support process-slot compaction handling and conditional process-status waiting.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/multiprocess/multiprocess_memory_limit.c`:
- Around line 69-103: Synchronize slot lifetime with lock-free readers by
changing get_current_proc_slot to return a pinned/read-guarded slot handle
rather than a raw pointer, while synchronizing access to region_info.my_slot. In
src/multiprocess/multiprocess_memory_limit.c lines 105-109, 485-491, 560-566,
and 1386-1390, acquire the guard and hold it through the respective status
read/store or accounting update, then release it. At lines 725-742, 932-937, and
951-956, perform slot retirement and compaction under the same reader-drain
protocol, retiring source slots only after active readers have exited.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e9350ed2-ba7e-45e5-89d8-5d7b5a55483e

📥 Commits

Reviewing files that changed from the base of the PR and between 52f33fc and 960cbe2.

📒 Files selected for processing (3)
  • src/include/memory_limit.h
  • src/multiprocess/multiprocess_memory_limit.c
  • src/multiprocess/multiprocess_memory_limit.h

Comment thread src/multiprocess/multiprocess_memory_limit.c
Validate cached process slots against the active shared-region range and clear the source slot after it is moved during compaction. Avoid waiting on process status when SM limiting is disabled.

Signed-off-by: Kizyulya Anatoliy <tolik8621@list.ru>
@AnatoliyKizyulya
AnatoliyKizyulya force-pushed the fix/process-slot-compaction branch from 9d64a0b to 025f456 Compare July 30, 2026 14:46

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
src/multiprocess/multiprocess_memory_limit.c (2)

69-103: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Stale cache never gets refreshed after cross-process compaction.

When another process compacts this slot away, the fallback loop deliberately leaves region_info.my_slot unrefreshed "Do not rewrite my_slot here: other threads may still be using the stale cache." That means every subsequent call from this process (a hot path used by memory accounting and status checks) pays an O(proc_num) linear scan for the remainder of the process's life, since nothing else ever repairs the cache for this process. Consider refreshing the cache with an atomic pointer store (atomic_store_explicit on region_info.my_slot as _Atomic(shrreg_proc_slot_t*)) instead of leaving it permanently stale — that resolves both the thread-safety concern noted in the comment and the perpetual fallback cost.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/multiprocess/multiprocess_memory_limit.c` around lines 69 - 103, Update
get_current_proc_slot so that, after the fallback scan finds the current process
slot, it refreshes region_info.my_slot via an atomic pointer store using the
field’s _Atomic(shrreg_proc_slot_t*) type. Preserve the existing validation and
return behavior while ensuring subsequent calls avoid repeatedly scanning
proc_num entries.

932-937: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Duplicated compaction block — extract a helper.

The 5-line sequence (copy_proc_slot_atomic + region_info.my_slot fixup + clear_proc_slot_atomic) is repeated verbatim for the PID=0 branch and the dead-proc branch. Extracting it reduces the risk of the two copies drifting apart on a future edit (e.g., the seqlock-protocol fix above would otherwise need to be applied twice).

♻️ Suggested helper
+static inline void compact_last_slot_into(shared_region_t* region, int target_slot) {
+    shrreg_proc_slot_t* last_slot = &region->procs[region->proc_num];
+    copy_proc_slot_atomic(&region->procs[target_slot], last_slot);
+    if (region_info.my_slot != NULL && region_info.my_slot == last_slot) {
+        region_info.my_slot = &region->procs[target_slot];
+    }
+    clear_proc_slot_atomic(last_slot);
+}

Then both branches reduce to region->proc_num--; compact_last_slot_into(region, slot); __sync_synchronize();.

Also applies to: 951-956

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/multiprocess/multiprocess_memory_limit.c` around lines 932 - 937, Extract
the duplicated compaction sequence into a helper such as compact_last_slot_into,
preserving the copy_proc_slot_atomic call, region_info.my_slot fixup, and
clear_proc_slot_atomic call. Replace the corresponding PID=0 and dead-proc
branches with region->proc_num--, the helper call, and the existing
synchronization, ensuring both paths share identical compaction behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/multiprocess/multiprocess_memory_limit.c`:
- Around line 725-743: Update clear_proc_slot_atomic to follow the seqlock
odd/even protocol used by add_gpu_device_memory_usage and
rm_gpu_device_memory_usage: mark slot->seqlock odd before clearing pid, status,
used[], device_util[], and related fields, then publish an even value only after
all clears complete. Do not expose the slot as stable while its fields are being
reset.

---

Nitpick comments:
In `@src/multiprocess/multiprocess_memory_limit.c`:
- Around line 69-103: Update get_current_proc_slot so that, after the fallback
scan finds the current process slot, it refreshes region_info.my_slot via an
atomic pointer store using the field’s _Atomic(shrreg_proc_slot_t*) type.
Preserve the existing validation and return behavior while ensuring subsequent
calls avoid repeatedly scanning proc_num entries.
- Around line 932-937: Extract the duplicated compaction sequence into a helper
such as compact_last_slot_into, preserving the copy_proc_slot_atomic call,
region_info.my_slot fixup, and clear_proc_slot_atomic call. Replace the
corresponding PID=0 and dead-proc branches with region->proc_num--, the helper
call, and the existing synchronization, ensuring both paths share identical
compaction behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 414a2d0b-c072-4a4f-915d-7f1f2e90831f

📥 Commits

Reviewing files that changed from the base of the PR and between 960cbe2 and 1641a52.

📒 Files selected for processing (3)
  • src/include/memory_limit.h
  • src/multiprocess/multiprocess_memory_limit.c
  • src/multiprocess/multiprocess_memory_limit.h

Comment thread src/multiprocess/multiprocess_memory_limit.c Outdated
@AnatoliyKizyulya

Copy link
Copy Markdown
Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
⚠️ Action not completed

No files to review.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@AnatoliyKizyulya

Copy link
Copy Markdown
Author

This patch also resolves the issue of running models on vLLM 0.27.1 with TP=2. It was tested on an H200 GPU using driver version 580.178.04.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/multiprocess/multiprocess_memory_limit.c`:
- Around line 932-937: Fix compaction ordering in
src/multiprocess/multiprocess_memory_limit.c#L932-L937 and `#L951-L956`: in both
cleanup branches, compute the last index without changing region->proc_num, copy
the last live slot, update region_info.my_slot when needed, publish the reduced
proc_num with a release store, then clear the source slot.
- Around line 1386-1390: Update wait_status_self so the no-slot path returns an
explicit result that ENSURE_RUNNING can distinguish from status mismatch, and
adjust the ENSURE_RUNNING loop to retry or handle that error rather than
treating -1 as success. Preserve the existing status comparison behavior when
get_current_proc_slot returns a valid slot.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 27020614-18f7-4c03-8f29-de98c3e400ee

📥 Commits

Reviewing files that changed from the base of the PR and between 1641a52 and 87385ce.

📒 Files selected for processing (1)
  • src/multiprocess/multiprocess_memory_limit.c

Comment thread src/multiprocess/multiprocess_memory_limit.c
Comment thread src/multiprocess/multiprocess_memory_limit.c
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Signed-off-by: Kizyulya Anatoliy <tolik8621@list.ru>
@AnatoliyKizyulya
AnatoliyKizyulya force-pushed the fix/process-slot-compaction branch from 87385ce to 48fd86b Compare August 12, 2026 14:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stale process-slot cache can stall TP=2 vLLM after shared-region compaction

1 participant